Exploded views for providing rich regularized geometric transformations and interaction models on content for viewing, previewing, and interacting with documents, projects, and tasks

ABSTRACT

The present invention relates to a system and method to facilitate efficient and automated presentation of information to users. A decomposition component automatically decomposes an information item into sets of subcomponents in a two- or three-dimensional isometric space, and generates visualizations having interactive graphics that allow users to inspect respective subcomponents. Interface features are provided to enable hover, dwell, and clicking commands, for example, providing a variety of options to zoom in, or change configurations of the visualization in accordance with the users intentions or inferences about what they desire to see or inspect more closely. Beyond the use of decompositions into exploded views of the content of items typically viewed as singular documents, the methods can be applied to perform geometric transformations on visualizations of multiple windows and other resources based on content and activity to provide regularized visualizations of the multiple items. Such exploded views can provide visual geometric summaries of projects, and a set of handles into accessing more detail on the subcomponents of the project.

REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of U.S. patent application Ser. No.10/609,890, filed Jun. 30, 2003 and entitled EXPLODED VIEWS FORPROVIDING RICH REGULARIZED GEOMETRIC TRANSFORMATIONS AND INTERACTIONMODELS ON CONTENT FOR VIEWING, PREVIEWING, AND INTERACTING WITHDOCUMENTS, PROJECTS, AND TASKS, which is incorporated herein byreference.

TECHNICAL FIELD

The present invention relates generally to computer systems. Moreparticularly, the present invention relates to systems and methodsproviding rich previews of documents, projects, and/or other digitallystored items via automated decomposition of such items in the form ofgraphical representations that are rendered by an associated userinterface.

BACKGROUND OF THE INVENTION

Various graphical user interfaces have been developed to provide aninteractive framework for computer users. Computer programs typicallyprovide a graphical user interface (GUI) to facilitate data entry, toenable viewing output on a display screen, as well as to manipulate orrearrange data. A graphical user interface can be associated with anapplication program or operating system shell, which may be running on auser's local machine and/or remotely, such as in a distributingcomputing system or over the Internet.

In view of continuing technological developments and increasing use ofthe Internet, people are using computers to access information to anever-increasing extent. Such information can reside locally on theperson's computer or within a local network or be global in scope, suchas over the Internet.

Users of window-based graphical user interfaces face difficult problemswhen they employ various programs for multiple tasks or activities—theyoften have a large number of windows to manage, with many windows foreach task. Switching between tasks is difficult because the windowsoften can be scattered around. Moreover, if windows are minimized whilenot in use, they are typically not organized together. If not minimized,a user can be faced with a difficult task of locating all relevantobscured windows and bringing them to a top of a display.

When users begin employing large display configurations (e.g., multiplemonitors), managing windows and tasks becomes an ever more difficultproblem, because minimized windows are kept in a location that may besignificantly distant from where they will be used. Managing manydisplay objects on small displays (e.g., PDA's) is also difficult—insuch case, oftentimes sufficient screen space is not available todisplay objects of interest.

As the amount of information content grows, another problem faced byusers is that they are often forced to perform combinations of searchingand browsing to identify information items of interest. Thus, users needmore efficient means to discriminate the target items they are pursuing.Also, as the complexity of each item grows, users may desire to havemore efficient access into portions of the items, without having tonecessarily open an item in a full-blown application in order to do so.Although various attempts have been made via conventional user interfaceschemes to address some of the aforementioned concerns, there is still asubstantial unmet need for a system and/or methodology that facilitatesefficient use of valuable computer user's time and cognitive resourcesin a multi-task working environment.

SUMMARY OF THE INVENTION

The following presents a simplified summary of the invention in order toprovide a basic understanding of some aspects of the invention. Thissummary is not an extensive overview of the invention. It is notintended to identify key/critical elements of the invention or todelineate the scope of the invention. Its sole purpose is to presentsome concepts of the invention in a simplified form as a prelude to themore detailed description that is presented later.

The present invention relates to systems and methods that generate richpreviews of files, and/or other digitally-stored items, in the form ofinteractive graphical representations of computational items or files,based on such properties of the item as type of item, item structure,item content, and metadata about the history of interaction with theitem, for example. The interactive representations allow users toinspect, probe, and navigate among document subcomponents of items atfocus of attention, before “launching” a full application.

One particular aspect of the present invention includes automateddecomposition of electronically stored items or sets of items intointerrelated subcomponents. This includes employing rich visualizationsand interactive graphics to expand the item or subcomponents intovarious “cognitive chunks” that can then be efficiently processed byusers. Respective systems and methods can be employed as a rich previewenabling users to inspect sets of items, such as text applications,presentation or graphics applications, and email documents, for example,within a rich “Exploded Views Previewer,” or interface that takes userswell beyond simple initial pages or other types of thumbnails. However,it is noted that preview applications in accordance with the presentinvention also can be generalized to other uses. As an example, richpreviewing can be employed within applications as a process forinspecting and navigating among components of an item being extended orrefined.

The subject invention also provides for a rich interface that greatlyfacilitates rapid viewing of information. Data, files, documents and soforth can be represented via 2 or 3-dimensional icons or display objectswherein a z-axis of the icon corresponds to parameter of the itemrepresented by the icon. As a user moves a cursor along the z-axis ofthe icon (or other axis), a preview pane displays for example pagescorresponding with the particular point(s) on the axis. A user can thusquickly review a document and locate a specific point in the item ofinterest. By selecting the point, the item is opened at that location ofinterest. The depth of the icon or display can also be indicative offile size or other predetermined metric.

To the accomplishment of the foregoing and related ends, certainillustrative aspects of the invention are described herein in connectionwith the following description and the annexed drawings. These aspectsare indicative of various ways in which the invention may be practiced,all of which are intended to be covered by the present invention. Otheradvantages and novel features of the invention may become apparent fromthe following detailed description of the invention when considered inconjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram illustrating item decomposition andrendering in accordance with an aspect of the present invention.

FIG. 2 is a diagram illustrating an exemplary exploded view interface inaccordance with an aspect of the present invention.

FIG. 3 is a block diagram illustrating subcomponent processing inaccordance with an aspect of the present invention.

FIG. 4 is a flow diagram illustrating exploded view processing inaccordance with an aspect of the present invention.

FIGS. 5-10 are example user interfaces illustrating exploded views anduser controls in accordance with an aspect of the present invention.

FIGS. 11-14 illustrate alternative example interfaces in accordance withan aspect of the present invention.

FIGS. 15-21 illustrate alternative exploded views in accordance with anaspect of the present invention.

FIG. 22 is a schematic block diagram illustrating a suitable operatingenvironment in accordance with an aspect of the present invention.

FIG. 23 is a schematic block diagram of a sample-computing environmentwith which the present invention can interact.

FIG. 24 illustrates an example application in accordance with an aspectof the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to a system and method to facilitateefficient and automated presentation of information to users. Adecomposition component automatically decomposes an information iteminto sets of subcomponents in a two- or three-dimensional isometricspace, and generates visualizations having interactive graphics thatallow users to inspect respective subcomponents. Interface features areprovided to enable hover, dwell, and clicking commands, for example,providing a variety of options to zoom in, or change configurations ofthe visualization in accordance with the user's intentions or inferencesabout what they desire to see or inspect more closely. In one example,the visualization and interactive behavior draw on an analogy to“exploded views” that are typically observed in engineering andtechnical manuals (e.g., an exploded view of a brake lining in anautomotive repair manual).

As used in this application, the terms “component,” “model,” “system,”and the like are intended to refer to a computer-related entity, eitherhardware, a combination of hardware and software, software, or softwarein execution. For example, a component may be, but is not limited tobeing, a process running on a processor, a processor, an object, anexecutable, a thread of execution, a program, and/or a computer. By wayof illustration, both an application running on a server and the servercan be a component. One or more components may reside within a processand/or thread of execution and a component may be localized on onecomputer and/or distributed between two or more computers.

As used herein, the term “inference” refers generally to the process ofreasoning about or inferring states of the system, environment, and/oruser from a set of observations as captured via events and/or data.Inference can be employed to identify a specific context or action, orcan generate a probability distribution over states, for example. Theinference can be probabilistic—that is, the computation of a probabilitydistribution over states of interest based on a consideration of dataand events. Inference can also refer to techniques employed forcomposing higher-level events from a set of events and/or data. Suchinference results in the construction of new events or actions from aset of observed events and/or stored event data, whether or not theevents are correlated in close temporal proximity, and whether theevents and data come from one or several event and data sources.

Referring initially to FIG. 1, a system 100 illustrates itemdecomposition and rendering in accordance with an aspect of the presentinvention. One or more electronic items 110 are processed by anautomated decomposition component 120 which formats the items 110 into adata storage 130. The formatting includes creation of one or moreprocessed items 140 through 150 which can then be presented and renderedto a user via a user interface 160. The automated decompositioncomponent 120 includes various decision processes or algorithms forformatting the processed items 140-150 and are described in more detailbelow with respect to FIG. 3. The processed items 140-150 include one ormore subcomponents that compose a grouping or subset of high-levelinformation displays that can be dynamically rendered to the user viathe user interface 160. Such renderings can include various dimensions,shapes, user controls, sizing, groupings, content renderings, and otherutilities for interacting with the subcomponents of the processed items140-150 and controlling the display of information to the user.

At preview time at the user interface 160 (or any time the systems andmethods of the system 100 are invoked), an item at focus is rendered ina rich geometrical layout, employing renderings and animations employingtwo- or three-dimensional graphics. The visualizations are generally afunction of one or more of properties associated with the type of item,item structure, item content, and metadata about the history ofinteraction with the item. Users can browse components of the items, attimes, selectively zooming with a mouse and keyboard (or other inputdevice) on subcomponents, in a graphical and/or semantic manner, andalso executing more traditional applications in new ways. As an example,a user can see visually, the last page that was edited and can bringthat page up to the immediate foreground in a Word processor via use ofthe user interface 160 having preview options for the processed items140-150 as is described in more detail below.

Beyond the basic design of user previews of the processed items 140-150,and associated set of interaction behaviors, a development environmentor SDK allowing third-parties to design and test different previewvariants for use in a more general operating system platform can beprovided. Such environment could be provided as a software package thatenables users to input or analyze desired electronic items 110 whileviewing and or manipulating various renderings of the processed itemsfor later display to the user. Also, users can be provided with a set ofpreference controls that could change, by type of item 110, the richpreview visualizations and access behaviors associated therewith. Thesystem 100 can also be coupled with offline, or real-time analysis(using principles of continual computation), and caching of the renderedresults so as to minimize latencies in real time.

It is to be appreciated that the present invention can employsubstantially any coordinate system, including multidimensionalcoordinate systems, and employ substantially any display format for theuser interface 160, wherein the display format can include substantiallyany shape, color, sound, dimension (e.g., displaying list of items in 3dimensions), code format—including embedded executables, and includecombinations of these and other respective formats or attributes. Inaddition, information retrieved from the data store 130 can be directedto substantially any portion of a display (not shown) associated withthe user interface 160. It is noted that display content can betransformed as it is rendered at the user interface 160. For example,the content or processed items—140-150 could be scaled in a smallermanner such as generating an iconic representation of the content.

The user interface 160 can be provided as part of the graphical userinterface in association with the data store 130 and can be provided aspart of and/or in association with a display. The display can beconfigured via interfaces regarding various aspects of display orcontent preferences, configurations and/or desired informationformatting by the user. The display can include display objects (e.g.,icons, buttons, sliders, input boxes, selection options, menus, tabs,and so forth) having multiple dimensions, shapes, colors, text, data andsounds to facilitate optimal control, sizing/resizing, format and/ordisplay of the processed items 140-150. In addition, various menus andalternative screens or display outputs can be provided that perform aplurality of aspects of the present invention. These aspects can alsoinclude a plurality of inputs for adjusting and configuring one or moreaspects of the present invention. This can include receiving usercommands from a mouse, keyboard, speech input and/or other device toeffect operations of the display via an associated graphical userinterface.

Referring briefly to FIG. 2, a system 200 is illustrates an exemplaryexploded view interface in accordance with an aspect of the presentinvention. In this aspect, a collection of item subcomponents 210 (e.g.,pages of a text document) are rendered in various formats at a display220. This can include rendering portions of a document and/or selectingvarious subcomponents and portions relating to a selected subcomponent.These aspects are described in more detail in the discussion relating toFIGS. 5-10.

The system 200 depicts a high-level example of the generation of anexploded view of a document. In today's systems, the initial documentmight appear as a text identifier or thumbnail. An exploded viewpreviewer 220 may structure the document in an isometric three-spacerepresentation, decomposed into a set of pages comprising the document,sequenced from front to back. Key pages, e.g., the initial page of thedocument might be further “exploded,” highlighting key content, such asfigures, graphics and links. Special pages, like the last page that wasedited or pages where most of the recent work has occurred may behighlighted via overall enlargement, being pulled out of a stack in oneor more ways. Such pages might also be decomposed via highlighting intocomponents that were pre-existing and components that were lastgenerated (red greeking). Users can mouse over different regions toexpand, move, and inspect additional details. Clicking on components orsub-details can invoke the appropriate application software to executeand bring the document up at a particular place noted by the user, viathe exploded view display 220.

Turning to FIG. 3, a system 300 is illustrates subcomponent processingin accordance with an aspect of the present invention. As noted above,an automated decomposition 304 may occur for one or more informationitems 308. For digital items, the structure of the “exploded view” canhinge on an analysis of the nature of the type of items at focus ofattention at 312 (e.g., is this a Powerpoint file, a Word document, aset of interrelated emails, a set of pictures, etc.). The structure ofthe document at 320 (e.g., number and sequence of pages, figures appearon a subset of pages, interrelated emails, sequence of images taken overtime, etc.), details about the content of the items at 330, and metadatacapturing such information as the when and/or how often differentcomponents were created, modified, or accessed at 340.

As an example, a rich exploded view preview might be configured todisplay when a preview is requested of a document, all pages of thedocument, in a sequence of pages in an isometric layout in apseudo-three dimensional space, using depth cues, and including thegraphical content of each page. The last page examined or modified bythe user can be specially rendered in a manner than points thissubcomponent out to the user to facilitate efficient processing ofinformation.

It is noted that the systems and methods of the present invention can beextended with rich metadata schema associated with documents or projectsthat captures such information as the time, nature, and duration ofaccesses. For example, for documents, “attentional annotations” can bestored as metadata capturing for different parts of the document, howlong a user was noted to have viewed or to have actively worked on thecomponents. The exploded view can use such information in decisionsabout the geometric layout and highlighting of the information,including the explicit use of metaphors like color and size to showregions that have been most attended to most recently, or that have beenattended to varying degrees over the lifetime of the document orproject.

FIG. 4 illustrates a methodology for exploded view processing inaccordance with the present invention. While, for purposes of simplicityof explanation, the methodology is shown and described as a series ofacts, it is to be understood and appreciated that the present inventionis not limited by the order of acts, as some acts may, in accordancewith the present invention, occur in different orders and/orconcurrently with other acts from that shown and described herein. Forexample, those skilled in the art will understand and appreciate that amethodology could alternatively be represented as a series ofinterrelated states or events, such as in a state diagram. Moreover, notall illustrated acts may be required to implement a methodology inaccordance with the present invention.

Proceeding to 402, one or more information items are automaticallyanalyzed. As noted above, this can include analyzing the types of itemsthat are at focus of attention, document structures, item content,and/or metadata associated with an item. At 404, an information itemsuch as a graphics file, presentation document, spreadsheet, worddocument, and so forth are automatically decomposed into one or moresubcomponents. At 408, item subcomponents are rendered in a graphicalinterface. Such interface can include multidimensional renderings toillustrate the respective subcomponents of the information item. At 412,user intentions and/or actions are automatically monitored in order tosuitably render selected subcomponents. For example, actions such asmouse clicks or curser hovering over an item can bring up more detailsrelating to the selected item or spawn other actions such as launchingan associated application.

FIGS. 5-10 are example user interfaces illustrating exploded views anduser controls in accordance with an aspect of the present invention.FIG. 5 depicts an information item 500 having one or more subcomponentswhich are graphically displayed along an axis of rendering 510.Respective subcomponents may have various portions such as illustratedat 520 and 530 for providing more detailed information such as graphics,text, embedded audio and/or image files, and so forth. It is to beappreciated that although rectangular components are illustrated,information can be rendered in substantially any size shape, color,dimension, and so forth. If a mouse selection is made at 540, anexploded view of the item 500 may appear as depicted in FIG. 6.

FIG. 6 depicts various subcomponents of an item 600 which can beselected and exploded to provide more detailed information, if desired.For example, portions 610 and 620 are expanded from a page to illustrateinformation such as graphics or text. If a mouse selection is made at630, another subcomponent is selected as depicted in FIG. 7.

FIG. 7 depicts a subcomponent 700 that has been selected by a user inFIG. 6. Upon selection, the subcomponent 700 can be raised orautomatically moved along any axis (according to user preferencesettings) to display information associated with the selection. At 710,the user may hover over a portion of the subcomponent 700 to furtherdrill for information that is depicted in FIG. 8.

FIG. 8 illustrates a selected portion 800 that is exploded to providefurther information. At 810, another subcomponent is selected by a userand is depicted in FIG. 9. In this Figure, the selection at 810 of FIG.8 causes a portion of another subcomponent to be displayed at 900 inFIG. 9. If desired, this portion can be further expanded via a selectionat 910. This expansion is depicted at 1000 of FIG. 10.

FIGS. 11-14 illustrate alternative example interfaces in accordance withan aspect of the present invention. FIG. 11 depicts an example interface1100 representing an exploded view preview of Word documents. Thedocument pages are rendered as a set of thumbnails in an isometricspace. Users can thumb through the pages of the preview by cursoringover the pages. Each page lifts out of the stack in turn, and pauses,given a cursor dwell, revealing, in a separate window 1110 more detailsabout the content of the page. Users can execute a Word application fromthe preview (e.g., MS Word, Word Perfect), bringing the applicationdirectly to the page for editing. FIG. 12 illustrates similar conceptsexcept that exploded views are applied to presentation documents such asPowerPoint, for example.

FIG. 13 is an example interface 1300 illustrating exploded view previewsfor email messages. Expanding out an email message reveals the structureof a conversation via a thread or other component. Useful annotationscan be added to the messages. Such annotations include indications ofpeople being added or dropped as the conversation has progressed. Otheraspects including changing color or sounds based upon characteristics ofa selected or displayed item. For example, (plus) +Icon in green for anew person in a user's email system and a (minus) −icon in red for adropped person. Users can navigate directly to the portion of theconversation of interest, appearing in an expanded text box at the rightor other location, via pointing at particular pages with a cursor.

FIG. 14 illustrates an example interface 1400 depicting exploded viewpreviews for email messages demonstrating a more complicated, branchingconversation. In this case two sub-threads or sub-conversations havebeen spawned from an initial message. As indicated by the people iconsin the highlighted message at 1410, people have been added and droppedin one of the initial sub-conversations. By hovering on the icons, asummary of the people added and dropped can be inspected. New people anddropped people can also be highlighted in the expanded text view at theright.

The systems and methods of the present invention apply not only topreviewing documents, before access, but also to visually summarizingdocuments or projects in other usage scenarios. A class of scenariosthat can harness the richness of the visual representations of explodedviews are task management and recovery scenarios and associated methods.Today, multiple resources and windows associated with different projectsare simply overlayed or manually opened and closed. Rich task managerscan be constructed that allow sets of resources, and associated displaysof content (e.g., windows displaying content and their positions) to bedefined as tasks and manipulated together, allowing for example theclosing and opening of tasks or “projects” involving the re-generationof windows with their content at the locations they were in before aproject was closed.

A task management and recovery system can enable users to define tasksor projects with content from one or more applications, eitherexplicitly, based on a task definition procedure such as an interfacethat allows a user to manually enter a set of resources, or implicitlyby linking tasks to particular applications (such as working with anemail application), or by observing which resources and applications areused together. Mixes of implicit or explicit definitions include themethod of allowing a user to invoke a “Save as Project” operation thatlists all recently touched or opened resources, including windows andthe content they are rendering that is currently being displayed on auser's screen. Such a Save as Project operation can allow a user to addnew resources or to remove resources that have been assumed as part ofthe task.

A system is provided that enables for definitions of persistent“projects,” that are comprised by one or more computational andassociated display resources, can allow users to open, close, and swapprojects. Exploded views can be valuable as regularized processes toclean up and redisplay projects that are revisited-or reconfigured andnormalized with an explicit “Clean up project,” procedure. Such aprocedure may be invoked as a menu item or button. Another applicationof exploded views is to provide a means for viewing the windows andother resources associated with a project, when such a project is storedfor later re-visitation, or is minimized into a small, iconicrepresentation of the multiple windows and other resources associatedwith the project. Exploded views for tasks or projects also allow easyreview of multiple projects by users, when a user may desire to view alist of stored projects. They can also allow for access to a preview ofa project in a way that allows them to access a particular portion ofthe project.

An example task management system (see related discussion at FIG. 24)enables a user to focus one or more full screens of a computer displaysolely on a single project by swapping, with simple gestures, theproject, along with the geometric layout of resources associated withthe project, into view and swapping out or minimizing other resourcesassociated with other tasks or projects. One approach to swapping out atask is simply to keep a label for the swapped out task in some visualrepresentation of a list of swapped out or stored tasks. Anotherapproach is to use a richer representation of the project. In oneapproach, an iconic, smaller view of the windows and other resourcesdisplayed in the last view of a project is displayed on the periphery ofa screen as a view and access portal onto the project. One sample designenables users to swap a project in by clicking on the small iconicrepresentation of the project, swapping in the project to a completeview, and potentially swapping out the project that had been at focus,into another iconic view at the periphery.

Exploded views can be used to regularize the view on a project when ithas been swapped out of focus, making the smaller, iconic view moreusable and understandable because of its regularized, familiarproperties. When a task is brought back, it can be brought back in itsoriginal configuration, or swapped back in as a larger version of theregularized view as a more understandable, more familiar, “cleaned up”starting point. In a refinement of this methodology, a user, who hasbeen switching between two different projects, may desire to have anexploded view as the preferred view on the project when the task isminimized but may desire to have the project represented as it had beenlast configured geometrically manually when it was at focus, if theproject is brought back into focus with some time horizon (e.g., thesame day). However, a user may desire to have a means of specifyingthat, if a project has not been revisited within some time horizon, tohave the project brought back in a regularized manner, as a previouslyspecified or parameterized exploded view of the project.

As an example, consider a simple project defined as “working withemail.” It is noted that more complex projects can be handled in asimilar manner. FIGS. 15-21 illustrate the use of exploded views inaccordance with an aspect of the present invention, centering on taskmanagement and shifting among distinct tasks for the “working withemail” example. FIG. 15 illustrates an application in focus such as anemail application. FIG. 16 illustrates the display of the applicationafter several messages have been opened in working with the emailapplication displayed in FIG. 15. When the email task is at the focus ofattention, the user may be looking at an inbox and opening multiplemessages as illustrated in FIG. 16.

In one approach (e.g., per user set up of preferences regarding thehandling of email as a project), the messages are labeled internally perthe exploded views analysis to be ordered (Message 1, 2, 3) by thesequence of how they have been opened. When the email task is swappedout, an exploded view is created. An exploded view is automaticallypositioned to the side of the screen and minimized as illustrated inFIG. 17, per a user's previously specified preferences, in a regularizedmanner, as the Inbox at center, and to the side, the set of emailmessages that the user has opened within some time horizon, cascadedneatly to the side of the inbox, in the order that the user had read themessages (or automatically resorted into messages by time or otherproperty like priority of the messages, per user settable preferences).

The use of exploded views to portray a visual summary of multiplewindows and other resources when a task is out of focus or minimizedallows a user to interact with components, e.g., individual messagesseparately when they are small and to bring to focus particular items,e.g., email messages as illustrated in FIG. 18. Also, when swapping thewhole task back into focus, thus maximizing the task as illustrated inFIG. 19, the user may desire to have access to a gesture that willeither return the task to the way it was rendered when last at focus orbring back the neater, regularized exploded view as a starting point asillustrated in FIGS. 20 and 21.

With reference to FIG. 22, an exemplary environment 2210 forimplementing various aspects of the invention includes a computer 2212.The computer 2212 includes a processing unit 2214, a system memory 2216,and a system bus 2218. The system bus 2218 couples system componentsincluding, but not limited to, the system memory 2216 to the processingunit 2214. The processing unit 2214 can be any of various availableprocessors. Dual microprocessors and other multiprocessor architecturesalso can be employed as the processing unit 2214.

The system bus 2218 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, 11-bit bus, IndustrialStandard Architecture (ISA), Micro-Channel Architecture (MSA), ExtendedISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Universal Serial Bus (USB),Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), and Small Computer SystemsInterface (SCSI).

The system memory 2216 includes volatile memory 2220 and nonvolatilememory 2222. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer2212, such as during start-up, is stored in nonvolatile memory 2222. Byway of illustration, and not limitation, nonvolatile memory 2222 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable ROM (EEPROM), or flashmemory. Volatile memory 2220 includes random access memory (RAM), whichacts as external cache memory. By way of illustration and notlimitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), anddirect Rambus RAM (DRRAM).

Computer 2212 also includes removable/non-removable,volatile/non-volatile computer storage media. FIG. 22 illustrates, forexample a disk storage 2224. Disk storage 2224 includes, but is notlimited to, devices like a magnetic disk drive, floppy disk drive, tapedrive, Jaz drive, Zip drive, LS-100 drive, flash memory card, or memorystick. In addition, disk storage 2224 can include storage mediaseparately or in combination with other storage media including, but notlimited to, an optical disk drive such as a compact disk ROM device(CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RWDrive) or a digital versatile disk ROM drive (DVD-ROM). To facilitateconnection of the disk storage devices 2224 to the system bus 2218, aremovable or non-removable interface is typically used such as interface2226.

It is to be appreciated that FIG. 22 describes software that acts as anintermediary between users and the basic computer resources described insuitable operating environment 2210. Such software includes an operatingsystem 2228. Operating system 2228, which can be stored on disk storage2224, acts to control and allocate resources of the computer system2212. System applications 2230 take advantage of the management ofresources by operating system 2228 through program modules 2232 andprogram data 2234 stored either in system memory 2216 or on disk storage2224. It is to be appreciated that the present invention can beimplemented with various operating systems or combinations of operatingsystems.

A user enters commands or information into the computer 2212 throughinput device(s) 2236. Input devices 2236 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 2214through the system bus 2218 via interface port(s) 2238. Interfaceport(s) 2238 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 2240 usesome of the same type of ports as input device(s) 2236. Thus, forexample, a USB port may be used to provide input to computer 2212, andto output information from computer 2212 to an output device 2240.Output adapter 2242 is provided to illustrate that there are some outputdevices 2240 like monitors, speakers, and printers, among other outputdevices 2240, that require special adapters. The output adapters 2242include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 2240and the system bus 2218. It should be noted that other devices and/orsystems of devices provide both input and output capabilities such asremote computer(s) 2244.

Computer 2212 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)2244. The remote computer(s) 2244 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device or other common network node and the like, and typicallyincludes many or all of the elements described relative to computer2212. For purposes of brevity, only a memory storage device 2246 isillustrated with remote computer(s) 2244. Remote computer(s) 2244 islogically connected to computer 2212 through a network interface 2248and then physically connected via communication connection 2250. Networkinterface 2248 encompasses communication networks such as local-areanetworks (LAN) and wide-area networks (WAN). LAN technologies includeFiber Distributed Data Interface (FDDI), Copper Distributed DataInterface (CDDI), Ethernet/IEEE 1102.3, Token Ring/IEEE 1102.5 and thelike. WAN technologies include, but are not limited to, point-to-pointlinks, circuit switching networks like Integrated Services DigitalNetworks (ISDN) and variations thereon, packet switching networks, andDigital Subscriber Lines (DSL).

Communication connection(s) 2250 refers to the hardware/softwareemployed to connect the network interface 2248 to the bus 2218. Whilecommunication connection 2250 is shown for illustrative clarity insidecomputer 2212, it can also be external to computer 2212. Thehardware/software necessary for connection to the network interface 2248includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and Ethernet cards.

FIG. 23 is a schematic block diagram of a sample-computing environment2300 with which the present invention can interact. The system 2300includes one or more client(s) 2310. The client(s) 2310 can be hardwareand/or software (e.g., threads, processes, computing devices). Thesystem 2300 also includes one or more server(s) 2330. The server(s) 2330can also be hardware and/or software (e.g., threads, processes,computing devices). The servers 2330 can house threads to performtransformations by employing the present invention, for example. Onepossible communication between a client 2310 and a server 2330 may be inthe form of a data packet adapted to be transmitted between two or morecomputer processes. The system 2300 includes a communication framework2350 that can be employed to facilitate communications between theclient(s) 2310 and the server(s) 2330. The client(s) 2310 are operablyconnected to one or more client data store(s) 2360 that can be employedto store information local to the client(s) 2310. Similarly, theserver(s) 2330 are operably connected to one or more server datastore(s) 2340 that can be employed to store information local to theservers 2330.

FIG. 24 illustrates an example application in accordance with an aspectof the present invention. FIG. 24 illustrates a schematic representationof an aspect of a system 2400 that facilitates controlling displayobject behavior in accordance with the subject invention. A displaycomponent 2410 effects rendering of display objects on a display space(not shown). The display space is typically a screen in connection witha computing system or device (e.g., desktop computer, laptop computer,personal data assistant, wireless telephone, television, . . . ). Thesystem 2400 further comprises a focus area component 2420, a scalingcomponent 2430 and a behavior component 2440. It is to be appreciatedthat some or all of these components can be part of a single componentand/or comprise a plurality of sub-components to affect various aspectsof the subject invention. The focus area component 2420 provides fordefining a stable region or focus area. The focus area component 2420can also define a progressive shrink area which is typically locatedabout an outside periphery of the focus area.

The scaling component 2430 provides for selectively scaling attributesof a display object as a function of location of the display objectwithin the display space. For example, when display object(s) are withinthe focus area 2460 the display objects are displayed and behave asusual. When display objects are moved outside of the focus area 2460into the progressive shrink region 2470, the display objects can bereduced in size based on their location, getting smaller as they near anedge of the display surface so that many more objects can remainvisible. It is to be appreciated the display object(s) can be movedoutside of and into the focus area manually (e.g., via a user employinga mouse), and it is contemplated that display object(s) can beautomatically moved outside of and into the focus area by the system2400. Automatic action (e.g., relocating of objects) can be taken as afunction of the system 2400 inferring a user's intentions with respectto manipulating display object(s). With respect to taking automaticaction, machine-learning techniques can be implemented to facilitateperforming automatic action. Moreover, utility-based analyses (e.g.,factoring benefit of taking correct automatic action versus costs oftaking incorrect action) can be incorporated into performing theautomatic action.

The behavior component 2440 can provide for modifying behavior ofdisplay objects in accordance with the subject invention. For example,display object(s) located within the focus area 2460 can behave in astandard manner with full functionality. When display object(s) arelocated outside of the focus area 2460, functionality associated withthe display object(s) can be modified. For example, refresh rates of thedisplay object(s) can be modified as a function of proximity to thefocus area 2460 and/or to an edge of the display space. In other words,objects in the periphery can also be modified to have differentinteraction behavior (e.g., lower refresh rate, static, . . . ) sincethey may be too small for standard techniques. In addition oralternatively, the objects can fade as they move toward an edge—fadingincreasing as a function of distance from the focus area and/or use ofthe object and/or priority of the object.

Beyond smooth manipulation of such aforementioned homogenous graphicalproperties with moves to the periphery, the subject invention can alsofacilitate richer notions of compression with the diminishing of size ofobjects, including compression with selective elision of unimportantcomponents of an object and the selective sizing of multiple components,so as to maximize likelihood that the object would still be recognizablein its reduced formulation. For example, key headings and distinctivefigures may be reduced proportionally less than less distinctive objectswith the decreasing size of objects. Moreover, the number of displayobject(s) within the display space as well as within the respectivesections (e.g., focus area, and progressive shrink area) can be factoredinto modifying behavior of the display object(s). The behavior component2440 can apply any suitable number of and/or combination of metrics(e.g., processing overhead, display space, number of display objects,relative location of display objects, priority associated withrespective display objects, time of day, user state . . . ) inconnection with modifying display object behavior in accordance with thesubject invention.

As noted above, the invention also can provide for object occlusionavoidance within a display area (e.g., focus area, periphery orprogressive shrink area)—such aspect of the invention mitigates objectsfrom obscuring one another. An extension of this aspect relates tocluster occlusion avoidance which mitigates clusters from obscuringother clusters as well as mitigating merging (e.g., since clustering canbe proximity based) of clusters as a result of moving clusters.

The system 2400 also includes a data store 2441 that can be employed tostore information (e.g., historical data, user profile data, displayobject data, system data, state information, algorithms, databases,display object current and/or previous state data, user current and/orprevious state info. multiple user info., task-related data . . . ) inconnection with the subject invention.

The system 2400 can optionally include an artificial intelligence (AI)2444 that can facilitate automatically performing various aspects (e.g.,modifying behavior of display object(s), scaling of display object(s),changing size and/or location of focus areas, changing size and/orlocation of progressive shrink areas, changing geometries of respectivefocus areas, changing geometries of respective progressive shrink areas,turning on and off functionalities associated with display objects,focus areas, progressive shrink areas, side bars . . . ) of the subjectinvention as described herein. The AI component can optionally includean inference component that can further enhance automated aspects of theAI component utilizing in part inference based schemes to facilitateinferring intended actions to be performed at a given time and state.The AI-based aspects of the invention can be effected via any suitablemachine-learning based technique and/or statistical-based techniquesand/or probabilistic-based techniques. For example, the use of expertsystems, fuzzy logic, support vector machines, greedy search algorithms,rule-based systems, Bayesian models (e.g., Bayesian networks), neuralnetworks, other non-linear training techniques, data fusion,utility-based analytical systems, systems employing Bayesian models, . .. are contemplated and are intended to fall within the scope of thehereto appended claims.

Although for ease of understanding, only a single focus area,progressive shrink area and display area are shown, it is to beappreciated that multiples of any and all of these areas is contemplatedand intended to fall within the scope of the hereto appended claims. Forexample, more than one focus area can exist with a display space, andlocations of such focus areas can be defined as desired. Moreover,functionality of respective focus areas can differ respectively (e.g.,so as to optimize multi-tasking). Likewise, multiple progressive shrinkareas can be defined, and functionality (e.g., with respect to modifyingdisplay object behavior) can be varied as desired.

What has been described above includes examples of the presentinvention. It is, of course, not possible to describe every conceivablecombination of components or methodologies for purposes of describingthe present invention, but one of ordinary skill in the art mayrecognize that many further combinations and permutations of the presentinvention are possible. Accordingly, the present invention is intendedto embrace all such alterations, modifications and variations that fallwithin the spirit and scope of the appended claims. Furthermore, to theextent that the term “includes” is used in either the detaileddescription or the claims, such term is intended to be inclusive in amanner similar to the term “comprising” as “comprising” is interpretedwhen employed as a transitional word in a claim.

What is claimed is:
 1. A graphical user interface in a display,comprising: at least one display object for displaying componentportions of a file in respective formats of the component portions,wherein the component portions are capable of being associated with aplurality of formats, including a text format, a presentation format, agraphics format, an email file format, and a sound file format, thecomponent portions being associated with at least two formats of theplurality of formats including the text format, the presentation format,the graphics format, the sound file format, or the email file format andthe component portions having an initial component portion displayed onan x-axis and a y-axis, wherein: the y-axis is perpendicular to thex-axis within a graphical user interface; and the remaining componentportions are cascaded along a z-axis within the graphical userinterface, relative to the initial component portion; at least one usercontrol for retrieving portions of information from the componentportions cascaded along the z-axis within the graphical user interface;and a summary component to summarize documents or projects in a dynamicapplication, the summary component minimizes an application into asmaller rendering that is replaced at focus by another application thatis optionally minimized into an exploded view, the exploded view beingautomatically positioned to a side of a screen and minimized perspecified preferences, the exploded view including a set of messagesthat a user has opened within a time horizon, cascaded to the side of atask in an order that the user can read the messages.
 2. The graphicaluser interface of claim 1, the component portions when associated withthe email file format include indications of people being added ordropped as a conversation has progressed.
 3. The graphical userinterface of claim 1, the component portions being capable of changes tocolors or sounds based upon characteristics of a selected or displayedcomponent portion.
 4. The graphical user interface of claim 1, thecomponent portions when associated with the email file format enableusers to navigate directly to a portion of a conversation of interest.5. The graphical user interface of claim 1, the component portions whenassociated with the email file format include at least two sub-threadsor sub-conversations that have been spawned from an initial message. 6.The graphical user interface of claim 1, the messages are automaticallyresorted by time or priority of the messages, per user settablepreferences.
 7. The graphical user interface of claim 1, the explodedview allows a user to interact with components of a task separately whenthe components of the task are minimized and to bring to focus selecteditems.
 8. The graphical user interface of claim 1, the exploded viewprovides access to a gesture that returns a task to an initial renderingwhen last at focus or return the task to a starting point view.
 9. Thegraphical user interface of claim 1, further comprising a metadataschema that captures information including at least one of a time, anature, or a duration of a user's access.
 10. The graphical userinterface of claim 9, the schema associated with attentional annotationsthat are stored as metadata capturing for different parts of a document,how long a user was noted to have viewed or to have actively worked oncomponents of the document.
 11. The graphical user interface of claim 9,the schema information is employed in decisions regarding a geometriclayout and highlighting of information including an explicit use ofmetaphors to show regions that have been most attended to or that havebeen attended to in the lifetime of the document or project.
 12. Amethod of implementing a graphical user interface, the methodcomprising: displaying, via a display object, component portions of afile having separate respective formats corresponding to the respectivecomponent portions and an initial component portion oriented accordingto an x-axis and a y-axis, wherein the y-axis is perpendicular to thex-axis, wherein the remaining component portions cascade along a z-axisof display corresponding to the initial component portion within agraphical user interface, and wherein the component portions are capableof being associated with at least a text document, a presentationdocument, a graphics document, an email file, and a sound file, thecomponent portions being associated with at least two respective formatscorresponding to the text document, the presentation document, thegraphics document, the email file, or the sound file; summarizing, via asummary component, documents or projects in a dynamic application, thesummary component minimizing an application into a smaller renderingthat is replaced at focus by another application that is optionallyminimized into an exploded view including a set of messages that a userhas opened within a time horizon, cascaded to the side of a task in anorder such that the user can read the messages; and retrieving, via auser control, portions of information from the component portionscascading along the z-axis of display.
 13. One or more computer storagedevices having computer readable instructions stored thereon that uponexecution via a processor program a computer to implement the graphicaluser interface in accordance with claim
 12. 14. One or more computerstorage devices having computer readable instructions stored thereonthat upon execution via a processor program a computer to implement agraphical user interface, comprising: at least one display object fordisplaying component portions of a file, wherein the component portionsare capable of being associated with a plurality of file types,including a text document, a presentation document, a graphics document,an email file, and a sound file, and the component portions areassociated with at least two file types of the plurality of file typesincluding the text document, the presentation document, the graphicsdocument, the email file, or the sound file rendered along an axis ofdisplay within a graphical user interface; at least one user control forretrieving portions of information from the component portions renderedalong the axis of display; and a summary component to summarizedocuments or projects, the summary component to minimize an applicationinto a smaller rendering that is replaced at focus by anotherapplication that is optionally minimized into an exploded view includinga set of messages that a user has opened within a time horizon, cascadedto the side of a task in an order that the user can read the messages.15. The one or more computer storage devices as recited in claim 14, thesummary component to summarize documents or projects in a dynamicapplication.
 16. The one or more computer storage devices as recited inclaim 14, the exploded view being automatically positioned to a side ofa screen and being minimized per specified preferences.
 17. The one ormore computer storage devices as recited in claim 14, wherein at leastone of: the email file includes indications of people being added ordropped as a conversation has progressed; the email file changes colorsor sounds based upon characteristics of a selected or displayed item;the email file enables users to navigate directly to a portion of aconversation of interest; or the email file including at least twosub-threads or sub-conversations that have been spawned from an initialmessage.
 18. The method as recited in claim 12, the exploded view beingautomatically positioned to a side of a screen and being minimized perspecified preferences.
 19. The method as recited in claim 12, wherein atleast one of: the email file includes indications of people being addedor dropped as a conversation has progressed; the email file changescolors or sounds based upon characteristics of a selected or displayeditem; the email file enables users to navigate directly to a portion ofa conversation of interest; or the email file including at least twosub-threads or sub-conversations that have been spawned from an initialmessage.
 20. The method as recited in claim 12, wherein the explodedview provides for interaction with components of a task separately whenthe components of the task are minimized and to bring to focus selecteditems.